1. Field of the Invention
The present invention is generally directed to neural networks. More particularly, the present invention is directed to hardware implementations of neural networks.
2. Background Art
Neural networks attempt to achieve autonomous behavior—i.e., learn—based on a network of simulated neurons that are connected in a manner suggestive of connections between real neurons in humans. In humans, a first neuron may fire in response to an external stimulus. In response to the firing of the first neuron, other neurons connected to the first neuron may also fire.
Similarly, a first simulated neuron in an input layer of a neural network can become active (e.g., fire) in response to stimulus to the neural network. One or more simulated neurons connected to the first simulated neuron may become active (e.g., fire) in response to the activity (e.g., firing) of the first neuron. Whether the activity of the first simulated neuron causes other simulated neurons to become active is dependent on at least two factors: (i) a weighting function associated with a connection between the first simulated neuron and each other simulated neuron to which the first simulated neuron is connected; and (ii) the threshold activity level required to cause each other simulated neuron to become active.
In a phase-model neural network, each neuron is associated with not only an activity, but also a phase. A phase-model neural network typically includes an integer number of phases (e.g., 32 phases). At each time step of a neural simulation, the phase of a neuron is updated based on a probability distribution of phases.
An undesirable property of conventional implementations of phase-model neural networks is that the amount of memory needed to store the probability distribution of phases is proportional to the integer number of phases included in the model. This property is undesirable for several reasons. For example, in conventional phase-model neural networks, the only way to make the number of phases large is to dedicate a large percentage of hardware resources to storing the probability distribution of phases. As another example, in most dedicated hardware platforms, it is difficult to create an adjustable amount of memory to accommodate an adjustable number of phases used in a phase-model neural network.
Given the foregoing, what is desired are systems, apparatuses, and methods for implementing a phase-model neural network using a fixed amount of memory, regardless of the number of phases used in the phase-model neural network.